Papers by Md Arafat Sultan
Learning Cross-Lingual IR from an English Retriever (2022.naacl-main)
Copied to clipboard
| Challenge: | DR.DECR is a cross-lingual information retrieval system trained using multi-stage knowledge distillation (KD) DRDECR demonstrates superior accuracy over direct fine-tuning with labeled CLIR data. |
| Approach: | They propose a cross-lingual information retrieval system with multi-stage knowledge distillation . they teach powerful multilingual representations and CLIR by optimizing two corresponding KD objectives . |
| Outcome: | The proposed system is the best single-model retriever on the XOR-TyDi benchmark . it is based on a multi-stage knowledge distillation process that can be expensive . |
On the Importance of Diversity in Question Generation for QA (2020.acl-main)
Copied to clipboard
| Challenge: | In this paper, we show that textual diversity in automatic question generation is beneficial for downstream QA. |
| Approach: | They propose to use textual diversity to promote automatic question generation as a quality measure for QA. |
| Outcome: | The proposed measure of QG quality correlates well with evaluation on QA. |
Confidence-Weighted Token Set Cover for Early Hypothesis Pruning in Self-Consistency (2026.findings-acl)
Copied to clipboard
| Challenge: | Despite its simplicity and efficacy, the high token expenditure of self-consistency can limit its practical utility. |
| Approach: | They propose a fast weighted set cover algorithm that utilizes two lightweight indicators to prune intermediate hypotheses periodically. |
| Outcome: | The proposed method reduces self-consistency token efficiency by 10-35% on three math benchmarks. |
PrimeQA: The Prime Repository for State-of-the-Art Multilingual Question Answering Research and Development (2023.acl-demo)
Copied to clipboard
Avi Sil, Jaydeep Sen, Bhavani Iyer, Martin Franz, Kshitij Fadnis, Mihaela Bornea, Sara Rosenthal, Scott McCarley, Rong Zhang, Vishwajeet Kumar, Yulong Li, Md Arafat Sultan, Riyaz Bhat, Juergen Bross, Radu Florian, Salim Roukos
| Challenge: | Question Answering (QA) is a major area of research in Natural Language Processing (NLP) |
| Approach: | They propose a one-stop and open-source QA repository for question answering . it supports core QA functionalities like retrieval and reading comprehension . they say it will facilitate easy replication of state-of-the-art (SOTA) QA methods . |
| Outcome: | The proposed framework enables easy replication of state-of-the-art (SOTA) QA methods. |
Answer Span Correction in Machine Reading Comprehension (2020.findings-emnlp)
Copied to clipboard
| Challenge: | Existing MRC systems produce only partially correct answers when presented with answerable questions. |
| Approach: | They propose a method that yields statistically significant performance improvements over existing MRC systems. |
| Outcome: | The proposed method yields statistically significant performance improvements over state-of-the-art MRC systems in monolingual and multilingual evaluation. |
FIRST: Faster Improved Listwise Reranking with Single Token Decoding (2024.emnlp-main)
Copied to clipboard
| Challenge: | Existing listwise LLMs lack efficiency as they provide ranking output in the form of a generated ordered sequence of candidate passage identifiers. |
| Approach: | They propose a listwise LLM reranking approach that leverages the first generated identifier to obtain a ranked ordering of the candidates. |
| Outcome: | The proposed approach accelerates inference by 50% while maintaining robust ranking performance with gains across BEIR benchmark. |
Cross-Task Knowledge Transfer for Query-Based Text Summarization (D19-58)
Copied to clipboard
| Challenge: | Existing methods for summarization data corpora are limited to extractive and abstractive summarizing. |
| Approach: | They propose to use machine reading comprehension (MRC) and query-based text summarization to produce extractive and abstractive summaries from pre-trained MRC and MT models. |
| Outcome: | The proposed model outperforms existing methods on CNN/Daily Mail and Debatepedia datasets and can be used as a baseline for future systems. |
Multistage Collaborative Knowledge Distillation from a Large Language Model for Semi-Supervised Sequence Generation (2024.acl-long)
Copied to clipboard
Jiachen Zhao, Wenlong Zhao, Andrew Drozdov, Benjamin Rozonoyer, Md Arafat Sultan, Jay-Yoon Lee, Mohit Iyyer, Andrew McCallum
| Challenge: | Low-resource tasks such as semi-supervised sequence generation require expert knowledge and cost. |
| Approach: | They propose a method for semi-supervised sequence generation where few examples are too scarce to fine tune a model. |
| Outcome: | The proposed method can generalize better than its teacher to unseen examples on semi-supervised sequence generation tasks. |
Moving Beyond Downstream Task Accuracy for Information Retrieval Benchmarking (2023.findings-acl)
Copied to clipboard
Keshav Santhanam, Jon Saad-Falcon, Martin Franz, Omar Khattab, Avi Sil, Radu Florian, Md Arafat Sultan, Salim Roukos, Matei Zaharia, Christopher Potts
| Challenge: | Neural information retrieval (IR) systems have progressed rapidly in recent years . many IR benchmarks focus on downstream task accuracy, concealing costs incurred . |
| Approach: | They propose to include efficiency considerations on IR benchmarks to help drive progress . eral et al. propose to incorporate query latency and cost budgets into evaluation . |
| Outcome: | a new study shows that the best IR system varies according to how efficiency considerations are chosen and weighed . the proposed benchmarks would allow for more thorough exploration of possible system designs . |
Towards Robust Neural Retrieval with Source Domain Synthetic Pre-Finetuning (2022.coling-1)
Copied to clipboard
Revanth Gangi Reddy, Vikas Yadav, Md Arafat Sultan, Martin Franz, Vittorio Castelli, Heng Ji, Avirup Sil
| Challenge: | Existing neural IR systems rely on lexical matching for query-passage alignment, while masked language models use a dual encoder architecture to encode passages and questions into continuous vector representations. |
| Approach: | They propose to enhance the out-of-domain generalization of Dense Passage Retrieval (DPR) through synthetic data augmentation only in the source domain. |
| Outcome: | The proposed model outperforms existing models in in-domain and zero-shot evaluations on Wikipedia-based datasets. |
Not to Overfit or Underfit the Source Domains? An Empirical Study of Domain Generalization in Question Answering (2022.emnlp-main)
Copied to clipboard
| Challenge: | Existing approaches to limit overfitting of training domains are rooted in this problem . domain generalization (DG) seeks to train models on a small number of source domains . |
| Approach: | They propose to use knowledge distillation to train models on a small number of source domains to maximize their zero-shot out-of-domain utility. |
| Outcome: | The proposed model learns its source domains better and has better out-of-domain generalization . the proposed model outperforms existing approaches that aim to limit overfitting . |
GPT-too: A Language-Model-First Approach for AMR-to-Text Generation (2020.acl-main)
Copied to clipboard
Manuel Mager, Ramón Fernandez Astudillo, Tahira Naseem, Md Arafat Sultan, Young-Suk Lee, Radu Florian, Salim Roukos
| Challenge: | Existing approaches to generating text from AMRs focus on training sequence-to-sequence or graph-tosequent models on annotated data. |
| Approach: | They propose a strong pre-trained language model with cycle consistency-based re-scoring to generate AMR text. |
| Outcome: | The proposed model outperforms existing methods on the English LDC2017T10 dataset. |
Structured Chain-of-Thought Prompting for Few-Shot Generation of Content-Grounded QA Conversations (2024.findings-emnlp)
Copied to clipboard
| Challenge: | Structured chain-of-thought (SCoT) prompting is used to generate content-grounded multi-turn questions and answers with a large language model. |
| Approach: | They propose a structured chain-of-thought prompting approach to generating content-grounded multi-turn question-answer conversations with a pre-trained large language model. |
| Outcome: | The proposed approach increases agent faithfulness to grounding documents by 16.8% when used as training data. |
Multi-Stage Pre-training for Low-Resource Domain Adaptation (2020.emnlp-main)
Copied to clipboard
Rong Zhang, Revanth Gangi Reddy, Md Arafat Sultan, Vittorio Castelli, Anthony Ferritto, Radu Florian, Efsun Sarioglu Kayi, Salim Roukos, Avi Sil, Todd Ward
| Challenge: | Existing approaches to transfer learning target data to in-domain text . prior work has adapted pre-trained LMs to specific domains . |
| Approach: | They extend the vocabulary of a pretrained language model with domain-specific terms to create synthetic tasks that help it transfer to downstream tasks. |
| Outcome: | The proposed approaches show significant performance gains on extractive reading comprehension, document ranking and duplicate question detection tasks. |